Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

🐛 Deflake TestWorkspaceController #2687

Closed
wants to merge 1 commit into from

Conversation

ncdc
Copy link
Member

@ncdc ncdc commented Jan 26, 2023

Summary

Rather than using a destructive test that deletes the root shard, create a workspace with a location selector that initially matches nothing, then add a shard that will match.

NOTE: this deflakes but there is still a timing issue where the system can schedule a Workspace onto a shard that was just deleted (stale shard cache). That is being tracked separately in #2686

Related issue(s)

Fixes #2603

@openshift-ci
Copy link
Contributor

openshift-ci bot commented Jan 26, 2023

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please ask for approval from ncdc by writing /assign @ncdc in a comment. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@ncdc
Copy link
Member Author

ncdc commented Jan 26, 2023

Looking into issue with the sharded setup

Rather than using a destructive test that deletes the root shard, create
a workspace with a location selector that initially matches nothing,
then add a shard that will match.

Signed-off-by: Andy Goldstein <andy.goldstein@redhat.com>
@ncdc ncdc force-pushed the deflake-workspace branch from 3a8f3ee to b872cb2 Compare January 26, 2023 20:58
@@ -141,7 +144,7 @@ func TestWorkspaceController(t *testing.T) {
if workspace.Spec.Cluster == "" {
return false, fmt.Sprintf("spec.cluster is empty\n%s", toYAML(t, workspace))
}
if expected := previouslyValidShard.Spec.BaseURL + "/clusters/" + workspace.Spec.Cluster; workspace.Spec.URL != expected {
if expected := rootShard.Spec.ExternalURL + "/clusters/" + workspace.Spec.Cluster; workspace.Spec.URL != expected {
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not quite sure why/how this was passing before. We always set the workspace's URL to $shardExternalURL/clusters/$clusterName

@ncdc
Copy link
Member Author

ncdc commented Jan 26, 2023

#2689
/retest

@ncdc
Copy link
Member Author

ncdc commented Jan 26, 2023

I think this change might be causing the various failures in the sharded setup. Will debug tomorrow

@ncdc
Copy link
Member Author

ncdc commented Jan 30, 2023

/retest

@openshift-ci
Copy link
Contributor

openshift-ci bot commented Jan 30, 2023

@ncdc: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
ci/prow/e2e-sharded b872cb2 link true /test e2e-sharded

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@openshift-merge-robot openshift-merge-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Feb 5, 2023
@openshift-merge-robot
Copy link
Contributor

PR needs rebase.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@ncdc
Copy link
Member Author

ncdc commented May 19, 2023

@sttts going to close this. Feel free to carry it forward if/as needed.

@ncdc ncdc closed this May 19, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/workspaces needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

bug? flake? TestWorkspaceController/add_a_shard_after_a_workspace_is_unschedulable,_expect_it_to_be_scheduled
2 participants